Junnan Dong writes about the WFM (Wiki Foundation Model), a novel paradigm designed to improve how AI agents utilize complex knowledge through an agent-native representation called LLM Wiki. Moving beyond traditional sparse graph representations, this model couples dense document contexts with multi-layered topological linkages via a specialized "Wiki Graph" schema. To address scalability issues in large-scale commercial deployments, the authors introduce an infrastructural NCCL boundary exchange protocol that optimizes distributed training by bypassing CPU serialization and leveraging fixed-shape GPU-to-GPU collectives.
- achieves 10.5 times training acceleration on distributed clusters
- uses a query-conditioned attentive aggregation for rich wiki message passing
- includes explicit attention variance regularization to improve semantic density
- outperforms existing models across five long-term agent memory and multi-hop reasoning benchmarks
- achieves 10.5 times training acceleration on distributed clusters
- uses a query-conditioned attentive aggregation for rich wiki message passing
- includes explicit attention variance regularization to improve semantic density
- outperforms existing models across five long-term agent memory and multi-hop reasoning benchmarks
